Papers with human-centered evaluation metrics
Annotator-Centric Active Learning for Subjective NLP Tasks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Annotator-centric active learning addresses the high costs of collecting human annotations by strategically annotating the most informative samples. |
| Approach: | They propose annotator-centric active learning which incorporates an annotation strategy following data sampling to approximate the full diversity of human judgments. |
| Outcome: | The proposed approach improves data efficiency and performs well in annotator-centric evaluations. |